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1 Preliminaries 

It is known that in the Grobner-Shirshov basis theory, the Shirshov's Composition- 
Diamond Lemma [19] plays an important role. In the Composition-Diamond Lemma, the 
order is asked to be monomial. In this paper, we generalize the Composition-Diamond 
Lemma by replacing the monomial order for others. From this result, we show by direct 
calculations of compositions, that the presentation of the HNN extension is a Grobner- 
Shirshov basis under an appropriate ordering of group words, in which the order is not 
monomial order. By the generalized composition lemma, we immediately obtain the 
Normal Form Theorem for HNN extensions. In fact, this is the first time to find a 
Grobner-Shirshov basis by using a non-monomial order. 

HNN extensions of groups were first invented by Higman- Neumann- Neumann in 1949 
(|12]) and independently by P. S. Novikov in 1952 (see [16], [IT], [IH])- The Normal Form 
Theorem for certain HNN extensions of groups was first established by L. A. Bokut (see 
[2], [1| and see also K. Kalorkoti [14j). In general, the Normal Form Theorem for 
HNN extensions of groups was proved in the text of Lyndon-Schupp |15j . 

For the alternating group An, a presentation was given in the monograph of N. Jacob- 
son (see [13], p71). However, we still do not know what is the normal form for An- In 
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this paper, we give the normal form theorem of An with respect to the above presentation. 

We first cite some concepts and results from the literature. Let k he a field, k{X) the 
free associative algebra over k generated by X and X* the free monoid generated by X, 
where the empty word is the identity which is denoted by 1. For a word w G X*, we denote 
the length of w by deg{w). Let X* be a well ordered set. Let / = Xlaex* /('^)'^ ^ k{X), 
where /(a) G k, with the leading word /. We say that / is monic if / has coefficient 1. 
We denote suppf = {a e X*\f{a) 0}. 

Definition 1.1 (fT^. see also J^, f^) Let f and g be two monic polynomials in k{X). 
Then, there are two kinds of compositions: 

(1) If w is a word such that w = fb = ag for some a,b E X* with deg(f) + deg{g) >deg{w), 
then the polynomial (/, g)w = fb — ag is called the intersection composition of f and g 
with respect to w. 

(2) If w = f = agb for some a,b E X* , then the polynomial (/, (yf)^, = / — agb is called 
the inclusion composition of f and g with respect to w. 

In the above case, the transformation f i-* (/, ^f)^ = f — agb is called the elmination of 
the leading word (ELW) of g in f . 

Definition 1.2 (^Bj, J^, cf JJ^) Let S C k{X) and < a well order on X* . Then the 
composition {f,g)w is called trivial modulo S if {f,g)w = where each ai e k, 

ai,bi e X* and aiSibi < w. If this is the case, then we write 

{f,9)w = mod{S,w) 

In general, forp,q G k{X), we write 

p = q mod{S, w) 

which means that p — q = J^^^i'^i^i^i) where each G k, ai, bi G X* and aiSibi < w. 

Definition 1.3 (l^i, IS^, cf. IT^ ) We call the set S with respect to the well order " < "a 
Grobner-Shirshov set (basis) in k{X) if any composition of polynomials in S is trivial 
modulo S . 

Remark: Usually, in the definition of the Grobner-Shirshov basis, the order is asked to 
be monomial. 

A well order " < " on X* is monomial if it is compatible with the multiplication of 
words, that is, for u,v & X*, we have 

u > V ^ W1UW2 > W1VW2, for all Wi, W2 G X*. 

The following lemma was proved by Shirshov [19] for the free Lie algebras (with deg-lex 
ordering) in 1962 (see also Bokut [5]). In 1976, Bokut P specialized the approach of 
Shirshov to associative algebras (see also Bergman [T]). For commutative polynomials, 
this lemma is known as the Buchberger's Theorem (see pTO]), published in |11] . 
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Lemma 1.4 (Composition-Diamond Lemma) Let A = k{X\S) and " < " a monomial 
order on X* . Then the following statements are equivalent: 

(i) S is a Grobner-Shirshov basis. 

(a) For any f G k{X), 7^ / G Ideal{S) ^ f = asb for some s E S, a,b E X*. 
(Hi) The set 

Red{S) = {ue X*\u ^ asb,s e S,a,b e X*} 
is a linear basis of the algebra A. 

If a subset S of k{X) is not a Grobner-Shirshov basis, then we can add to S all nontrivial 
compositions of polynomials of S, and by continuing this process (maybe infinitely) many 
times, we eventually obtain a Grobner-Shirshov basis 5'™'"^. Such a process is called the 
Shirshov algorithm. 

If 5* is a set of "semigroup relations" (that is, the polynomials of the form u — v, where 
u,v E X*), then any nontrivial composition will have the same form. As a result, the set 
gcomp g^igQ consists of semigroup relations. 

Let A = sgp{X\S) be a semigroup presentation. Then 5* is a subset of k{X) and 
hence one can find a Grobner-Shirshov basis S^°"^'^. The last set does not depend on k, 
and as mentioned before, it consists of semigroup relations. We will call S'^""^'^ a Grobner- 
shirshov basis of A. This is the same as a Grobner-shirshov basis of the semigroup algebra 
kA = k{X\S). 

2 Generalized Composition-Diamond Lemma 

In this section, we generalize the Composition-Diamond Lemma which is useful in the 
sequel, by replacing the monomial order for others. The proof of the following lemma is 
essentially the same as in [7]. For the sake of convenience, we give the details. 

Lemma 2.1 (Generalized Gomposition-Diamond Lemma) Let S C k{X), A = k{X\S) 
and " < " a well order on X* such that 

(A) asb = asb for any a,b E X*, s E S; 

(B) for each composition {si,S2)w in S, there exists a presentation 

{si,S2)w = ^aiGitibi, QiUbi < w, where ti E S, ai,bi E X*, ai E k 

i 

such that for any c,dE X* , we have 

caiUbid < cwd (1) 

Then, the following statements hold, 
(i) S is a Grobner-Shirshov basis. 
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(a) For any f G k{X), 7^ / G Ideal{S) =^ f = ash for some s E a,b E X* . 
(Hi) The set 

Red{S) = {u E X*\u ^ ash, s E S,a,b E X*} 
is a linear basis of the algebra A. 

Proof (i) is clear. Now, we prove (ii). Let 

n 

f = ^aittiSibi, Ui E k, Si E S, ai,bi E X* 

i=l 

Assume that 

Wi = aiSih, wi = W2 = ■ ■ ■ = wi > wi+i ■ ■ ■ 

We will use the induction on / and Wi to prove that / = asb, for some s E S and a,b E X*. 
If / = 1, then by (A), / = aiSi^i and hence the result holds. Assume that / > 2. Then 

/ = («! + a2)aiSibi + a2(a2S2&2 - H 

For wi = W2, there are three cases to consider. 

Case 1. Assume that bi = and 02 = aiSib. Then we have 

a2S2&2 - aisibi = aiSib{s2 - 52)^2 - cn(si - Si)6s2&2- 

For any t E supp{s2 — S2), by (A), aiSibtb2 = aiSibtb2 < aiSi6s2&2 = wi and similarly, we 
have aiti6s2&2 < Wi, for any ti E supp{si — Si). 

Case 2. Assume that 61 = 662, ^2 = ciiO', Sib = as2 and degsi + degs2 > deg{asi). 
Then 

02^262 - aiSibi = ai{as2 - 316)62- 

By (B), there exist Pj E k, Uj,Vj E X*, tj E S such that UjtjVj < w = Sib, Sib — as2 = 
PjUjtjVj and aiUjtjVjb2 < aiSibb2. Now, by (A), for any j, we have aiUjtjVjb2 = 
aiUjtjVjb2 < aiSibb2 = Wi. 

Case 3. Assume that 62 = 661, 02 = aia and Si = 0^26. Then 

02-5262 — aiSibi = ai{as2b — Si)6i. 

by (A) and (B), there exist Pj E k, Uj,Vj E X*, tj E S such that UjtjVj < w = 
si, si — as2b = J2j Pj'UjtjVj and for any j, we have aiUjtjVjbi = aiUjtjVjbi < aiSibi = wi. 
(iii) follows from (ii). □ 

3 Normal form for HNN extension of group 

In this section, by using the Grobner-Shirshov basis, we provide a new proof of the normal 
form theorem of HNN extension of a group. 

Definition 3.1 (ll ^ . IT ^ , fl^, IT B^ ) Let G be a group and let A and B be subgroups of 
G with (j) : A B an isomorphism. Then the HNN extension of G relative to A , B and 
(f) is the group 

Q = gp{G, t; t^^at = (j){a), a E A). 
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Let G = Gi U {1}, where = G \ {1} = {g^; a e A}, and let 

G/A = {giA;i e I}, G/B = {h^B-j e J}, 

where {gi;i G /} and {hj;j e J} are the coset representatives of A and B in G, 
respectively. We assume that all sets A, /, J are well ordered and so are the sets 
{ga',Oi e A}, {gfji e /}, {hj;j e J}. Then we get a new presentation of the group 
^ as a semigroup: 

g = sgp{Gi,t,t-^ ; gg' = [gg'],gt = gj(j){ag), gt'^ = gj-^(j)-^{bg), 

ft-' = l, g,g'eG,, e = ±l), 

where [gg] e G; g = g^ag, g^ e {gi;i el}, g g^, ag e A; g ^ g^bg, g^ e {hfj e 
J}, 9 ^ 9b^ bg e B. 

Now, we order the set G in three different ways: 

(1) Let 1 < g^ < gg < ■ ■ ■ (a < /3) be a well order of G. Then we denote this order by 
(G, >) and call it an absolute order. 

(2) For any g,g' G G, suppose that g — g^^ag^g = g/^g'- Then g >^ g if and only if 

(S'a' ^g) > (S'l' ^g) is ordered lexicographically (elements g^iOj^ by /, elements ag, a^' 
by (1)). We denote this order by (G, > J and call it the A-order. In particular, if 
9^9a^ then g>^g^, for {g^, ag) > {g^, 1), Ug ^ 1. 

(3) For any g,g' G G, suppose that g — g^hg^g — g'J}g'- Then g >^ g if and only if 
{gg, bg) > {g'^, b'g) is ordered lexicographically (elements gB,g'g by J, elements bg, b^i 
by (1)). We denote this order by (G, >^) and call it the S-order. 

Then we order the set G\ in three different ways too: 

(1) The absolute order (Gi*, <) is deg-lex order, to compare words gi - ■ -gn {it- ^ 0) 
first by length and then lexicographically using absolute order of Gi. 

(2) The A-order (Gi*, <^) is deg — lex^ order, firstly to compare words fifi • • • g'n (^ > 0) 
by length, secondly for n > 1, (?!, ■ ■ ■ , gn-i lexicographically by absolute order, and 
finally the last elements gn by 74-order. 

(3) The S-order (Gi*, < b) is similar to (2) replacing >^ by >^. 

Each element in {GiiJ{t,t~^}}* has a unique form u = uit'^^U2t'^'^ • • •Ukt^'^Uk+i, where 
each Ui e Gl,ei = ±1, k> 0. Suppose that v = vit^'-V2 ■ ■ -vit^'vi+i e {GiU{t, 

Then 

wt{u) ^{k,f\--- , f*',ui, ■■• ,Uk, Uk+i) 

We define u >- v if wt{u) > wt{v) lexicographically, using the order of natural numbers 
and the following orders: 

(a) t > t-^ 
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(b) Ui >^ Vi if Ei = 1, 1 < i < k 

(c) Ui >g Vi if Ei = —1, 1 < i < k 

(d) Uk+i > {k = I), the absolute order of Gi* 

Now, we can easily verify the following lemma. 

Lemma 3.2 Let the order >- on {Gi(j{t,t^^}}* be defined as above. Then the order >- 
is a well order but not monomial, for example, g >~ g does not necessarily imply that 
gt y g't. 

Equipping with the above notation, we have the following lemma. 

Lemma 3.3 Let X = Gi(j{t,t^^} . Suppose that the order >- on X* is defined as above 
and S = {gg - \gg \ gt - gj(f){ag), gt'^ - g^t'^cp-^ihg), tH'" - 1 \ g,g' eGi, e = ±1} 
is as above too. Then S satisfies conditions (A)-(B) in Lemma l2J[ 

Proof For any c,d E X*, suppose that c = Cit^^ ■ ■ ■ Cnt^"Cn+i, d = dit^^ ■ ■ ■ dmt^"'dm+i, Ci, dj G 
Gl, Ei,6j = ±1. We firstly check (A). Then there are four cases to consider. For exam- 
ple, the second case is for polynomials gt — g^t(f){ag), g ^ g^- We need to prove that 
cgtd >- cgAt(j){ag)d for any c, d. 

Since 

wt{cgtd) = {n + m + l,f\--- ,f\t,t^\--- ,t^"^,ci,--- ,Cn,Cn+ig, 

diy ■ ■ ■ , dmy C^m+l)? 

wt{cgj(j){ag)d) = {n + m + 1,^^ ■ ■ ■ ,t^",t,t^\- ■ ■ ,t^"',Ci, ■ ■ ■ ,c„,c„+i5f^, 
(j){ag)di, ■ ■ ■ , djn, C?m+l) 

and Cn+ig >^ Cn+ig^ (since g >^ g^), we have cgtd >- cgAt(f){ag)d. 

We secondly check that (B) holds in Lemma 12.11 By noting that there is no inclusion 
compositions in S, we need only to consider the cases of intersection compositions. For 
any a, 6 G X*, Si,S2 G S, suppose that asi = S26 with degsi + degs2 > deg{asi). Then, 
we consider the following cases: 

w = gg'g", gg't {g' ^ g'J, gg't-' {g' ^ g'J, gtH'^, t^' {e = ±1). 

For example, the second case is as follows: 

Let si = gg', S2 = g't, w = gg't. Then, by noting that gg' = [gg'j^a^gg'] = 
[9g'jA%9'^]0'9' implies that [gg]^ = [gg'j^ and aj^^'j = a^gg'^Og,, we know that (s2, = 
[99']t - gg'j(PM = {[gg']t - [gg']j(p{a^gg'])) - {gg'^ - [gg'j)t(f){ag>) - {[gg Jt^{ag>) - 
[99'jAt<Pi%9'^](^g'))- We denote s[ = [gg']t - [gg']Mi%9'])^ 4 = 99'^ - N^], &2 = 
t(^{ag<), S3 = [gg ^t(l){ag,) - [gg 2\A^'^^%g'J^g')- Clearly, s'^, s'^, S3 G ^ or {0} (if 
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[gg] = [99] a)- Since 

wt{cs'^d) = {n + m + l,f\--- ,f",t,t^\--- , f^™, ci, ■ ■ ■ ,c„,c„+i[W], 

wt{cs'2b2d) = {n + m + l,f\--- ,f",t,t^\--- , f^", Ci, ■ ■ ■ , c„, c„+ic/^^, 

u;t(csV) = in + m + l,f\--- , t"^™, ci, ■ ■ ■ , c„, c„+i[^^^], 

wt{cwd) = {n + m + l,f\--- ,f",t,t^\--- ,t^^,ci,--- ,Cn,Cn+igg\ 

and Cn+igg' >^ Cn+i[gg'], Cn+igg'^, Cn+i[gg'j, we have cwrf y cs[d, cs2b2d, cs'^d. All 
other cases are treated the same. The proof is finished. □ 

Now, by Lemma 12.11 and Lemma 13. 3[ we obtain the following theorem. 

Theorem 3.4 A Grobner-Shirshov basis of HNN extention Q = gp{G,t]t~^at = (j){a),a G 
A) consists of the following relations: 

(3.1) gg'=[gg']; 

(3.2) gt = gj(f){ag), where g = g^ag, g^ G {gi,i G /}, G A; 

(3.3) gt~^ = gj-^(l)~\bg), where g = g^^bg, g^ G {hj.j G J}, bg G B; 

(3.4) tt-^ = 1, t-H = 1. 

where {gfji G /} and {hj\j G J} are the coset representatives of A and B = (f){A) in G, 
respectively. 

Thus 5* is a Grobner-Shirshov basis of the algebra /c(G'iU{t, t^^}) with 

Red{S) = {ue {GiUjt, t-^}Y\u ^ asb, seS,a,be {GiU{t, r^}}*} 

which is the normal form of the HNN extension Q. From this result, the normal form 
theorem for HNN Extension of the Group G easily follows. 

Theorem 3.5 (The Normal Form Theorem for HNN Extension, [TE^ . Theorem 4-2.1) 
Let Q = gp{G,t;t^^at = (f){a),a G A) be an HNN extension of group G. If {g^i G 
/}, and {hj-jj G J} are the sets of representatives of the left cosets of A and B = 
(f){A) in G, respectively, then every element w of Q has a unique representation w = 
gif^^ ■ ■ ■ gnt'^"gn+i > 0, ei = ±1), where, for I < I < n, the following conditions are 
satisfied: 

(1) if El = 1, then gi G {g^i G /}, 

(2) if El = -1, then gi G {hf,] G J}, 
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(3) there does not exist subwords tt and t ^t, 

(4) Qn+i is an arbitrary element ofG. 

Remark: In Theorem 4.2.1 of [15j, the right cosets were considered. We notice that the 
above Theorem 13.51 is essentially the same as Theorem 4.2.1 in [T5] . 

4 Normal form for alternating group 

In this section, we first find a Groner-Shirshov basis for the alternating group and 
then we give the normal form theorem of An- 

Let Sn be the group of the permutations of {1,2, ■ ■ ■ ,n}. Then the subset An of all 
even permutations in Sn is a normal subgroup of Sn- We call An the alternating group of 
degree n. The following presentation of An was given in the monograph of N. Jacobson 
(see [13], p71): 

An = gp{xi {I <i <n-2) ; xf = 1, (xi^iXi)^ = x'f = 1 {2 < i < n - 2), 

{x,xj)^ = l {l<t<J-l,J <n-2)) 

where Xi = (12) ((i + l){i + 2)), i = 1, 2, ■ ■ ■ , n — 2, (ij) the transposition. We now give 
a presentation of the group An as a semigroup: 

An = sgp{xi^, Xi (1 < i < n — 2); Xix^^ = x^^xi = a;'^ = 1, (xi^iXi)^ = xf = 1 
{2 <i <n-2), {xiXjf = 1 (1 < i < j - 1, J < n - 2)) 

We now order the generators in the following way: 

< Xi < X2 < ■ ■ ■ < Xn-2 

Let X = {x^^, Xi,X2, - ■ ■ , Xn-2}- Then, with the above notations, we can order the words 
of X* by the deg-lex order, i.e., compare two words first by their degrees, then order them 
lexicographically when the degrees are equal. Clearly, this order is a monomial order. 
Now we define the words 

•Av J 2 i-C J lAj J — . ]_ * * ■ I 5 

where j > i> 1 and Xji^ = XjXj^i...Xi^,e = ±1. 

The proof of the following lemma is straightforward. We omit the details. 
Lemma 4.1 For e = ±1, the following relations hold in the alternating group An: 

(4.1) xf =x^' 

(4.2) = 1, (z > 1) 

(4.3) XjXi = XiXj, {j — 1 > i > 2) 

(4.4) XjXl = x^'^Xj, (j > 2) 
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(4.5) XjiXj = Xj_iXji, {j > i> 2) 

(4.6) XjieXj = (j > 2) 

(4.7) X2Xi'^X2 = Xi^X2Xi^ 

(4.8) xfx^^ = 1 

Now, we can state the normal form theorem for the group An. 
Theorem 4.2 Let 

An = gp{xi {I <i <n-2) ; xl = 1, (xi^iXi)^ = Xi^ = 1 {2 <i <n - 2), 

{x,x,f = l (l<«<j-l,j<n-2)) 

be the alternating group of degree n. Let S = {xf^ —x^^, xf — l {i > 1), XjXi—XiXj (j — 1 > 

Z ^ 2), XjX-^ X-^ Xj (j ^ 2), XjiXj Xj — \Xji (j > i > 2), XjieXj - Xj-iXji^s {j > 

2), X2X1X2 — x]^^X2Xi^, x\xi^ — 1, e = ±1}, where xji = XjXj-i...Xi, j > i > 1, xne = 
XiXi-i-.-Xi"^ , e = ±1. Then 

(i) S is a Grobner-Shirshov basis of the alternating group An; 

(a) every element w of An has a unique representation w = xij-^X2j2---Xn-2j„-2> where 
xtt = xt {t > 1), Xii+i = 1, Xii = XiXi-i ■ ■ - xl, I < ji < i + l, I <i < n-2, e = ±1 
(here we use Xn instead of Xn^). 

Proof By Lemma 14.11 it is easy to see that every element w of An has a representation 
w = xij^X2j2---Xn-2jn-2 {ji < + !)• Here xij^ may have 3 possibilities X2j2 4 

possibilities, and generally, Xjj. i + 2 possibilities, 1 < i < n — 2. So there are n\/2 words. 
From this fact, it follows that each representation is unique since \An\ = n\/2. On the 
other hand, it is clear that Red{S) consists of the same words w. Hence, by Composition- 
Diamond Lemma, 5* is a Grobner-Shirshov basis of the alternating group An. □ 

Remark: According to normal form in Sn-i is Xij-i^X2j2---Xn-2jn-2: but here Xn = 
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